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When designing a program to improve the critical 
thinking skills of students, it is important to identify the specific 
learning outcomes for which to strive. In addition, the instructor 
roust determine why students are not currently exhibiting the kind of 
critical thinking skills desired in order to develop the appropriate 
techniques and methods to overcome these skills deficiencies. 
Attempts to enhance critical thinking and reasoning skills have 
generally fallen into two categories: stand-alone courses, and 
discipline-specific courses in which thinking enhancement efforts 
have l>een incorporated. Stand-alone courses tend to be less effective 
because the skills acquired must then be transferred to subsequent 
courses. The three steps to selecting or creating a program for 
enhancing students' thinking and reasoning skills are as follows; 
determine long-term goals? (2) assess the needs and characteristics 
of students; and (3) identify short-term tactical goals which can be 
realistically attained in the time allotted. Following a detailed 
listing of critical thinking skills and a review of program 
development strategies, this report provides brief descriptions of 13 
specific programs including the following: (1) Accent on the 
Development of Abstract Processes of Thought (University of 
Nebraska-Lincoln); (2) Development of Operational Thinking Skills 
(Illinois Central community College); (3) Development of Reasoning in 
science (California State University at Fullerton) ; (4) Mathematical 
Preparation for Physics (University of Alabama at Birmingham); and 
(5) A Practicum in Thinking (University of Cincinnati, Ohio). A 
30-iteni bibliography is included. (PAA) 
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SABBATICAL LEAVE REPORT 

William Kemler 

January 8, 1990 

Suggested Guidelines for Selecting or Creating Programs 
to Enhance Thinking and Reasoning Skills 

Introduction 

The educational community has embarked upon a crusade to Improve the 
thinking and reasoning skills of students. Teachers are perplexed by the tell- 
me-what-l-need-to-know-so-i-can-glve-lt-back-to-you-on-a-test approach to 
learning of many students and the low priority given to understanding that 
results. Scientists are dismayed by the fact that Important discoveries which 
will profoundly change the lives of people are not understood by the general 
public. Businesses are worried about the quality of the future work force. 
Political leaders are concerned about the ability of the electorate to make 
reasoned, democratic decisions in the face of complex political and social Issues. 
Recent surveys show that large numbers of people are turning sway from logic 
and reason in favor of dogmatic and illogical belief systems. 

rnncern about the quality of thinking Is nothing new, of course. Educators 
in this country have lamented a lack of critical thinking on the part of 
students since the time of William James. It is only the sense of urgency and 
the magnitude of the concern that have changed recently. There seem to be 
twc major reasons for these changes. First, more people are expected to have 
higher-level thinking skills in the "Hi-tech" age of the 1990's. The average 
person has higher educational aspirations than in the past, and higher standards 
of literacy are expected by society (take a look at the manual that came with 
your VCR or the instructions for filling out a tax return). College is no 
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longer reserved for the social elite and those of above-average ability. 
Post-secondary education Is Increasingly becoming the prerequisite for a decent 
job. Second, Increased competition from countries with better-educated workers 
has forced many businesses to acknowledge that a work force capable of making 
decisions and solving problems on the spot Is more efficient and productive than 
one which has to wait for Instructions to filter down from management. The 
business community has begun to put political pressure on the government to 
support educational reform and has helped to focus the public spotlight on 
deficiencies In our educational system. 

The original Intent of my leave was .to review theoretical and empirical 
work In the area of thinking and reasoning and then evaluate specific programs 
designed to enhance these skills at the college level. I was particularly 
Interested In Identifying programs that might be usefully Incorporated Into the 
curriculum at LLCC. Upon completing my review of the background literature, 
I realized that making general pronouncements about the suitability of programs 
for either general curriculum development or Incorporation into specific classes 
would be of little value without detailed information about program goals and 
the needs of specific instructors, a bit like recommending a new car to a 
friend. 

The difficulty stems from the fact that in order to evaluate the success of 
a program, one must compare outcomes to goals. Goals vary from program to 
program according to Instructional needs, and Instructional needs are perceived 
differently by different educators. What exactly do we want students to be 
able to do when they engage in "critical thinking"? After reviewing the 
literature and discussing the matter with colleagues from LLCC and other 
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colleges and universities, I discovered that, depending upon whom you talk to. a 
critival thinker should be able to: 

actively process and evaluate information 
recognize source of bias 

look for both the practical and theoretical implications of information 
reject overly simplistic, dualistic thinking 
continually refine hypotheses 
examine all sides of an issue 
Identify and question assumptions 

analyze the meaning of important terms and concepts and examine the way 
in 

which terms are defined 
question generalizations and conclusions 
suspend judgment until there is adequate evidence 
consider and explore alternative explanations 

draw appropriate conclusions based on reason and evidence and be able to 
defend them 

examine and evaluate one's own values and judgments ^ 
recognize and overcome one*s own biases 
question cherished beliefs when evidence contradicts them 
look beyond the obvious 
question authoritative pronouncements 
apply appropriate problem-solving and reasoning skills 
examine ail the evidence available (avoid selective perception) 
maintain a sense of healthy skepticism 
be aware of the Importance of context 
recognize Important variables In a situation 
summarize or paraphrase the work cf others in one's own words 
be aware of the g^s in one's own knowiedgs 
revise and reorganize thoughts in light of new information 
understand the need for operational definitions 
translate back and forth between words and symbols 
discriminate between observation and In'arenc* 
discriminate between Inductive and deductive reasoning 
vorlfy inferences, conclusions or results 
be aware of one's own thinking and reasoning processes 
compare and contrast different descriptions or explanations of the same 
events 

distinguish betwaen relevant and Irrelevant Information 
gather and synthesize relevant Information 
Identify misinformation, disinformation and r .'ejudlce 
avoid making judgments on the basis of emotion or expediency 
seek out and consider a wide variety of points of view 
clarify Issues and arguments and break down complex Ideas into simpler 
parts 

evaluatk) the credibility of sources 

transfer Ideas to new contexts and make Interdlsclpllndry connections 
recognize and apply abstract principles In actual situations 
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tolerate unavoidable ambiguity and come to terms with uncertainty and 

Inconsistency 
detect logical fallacies 

synthesize component ideas into new arrangements 

recognize underlying similarities In superficially different contexts 

separate fact from opinion 

test hypotheses . 
reason beyond one's own Interest (value truth more than power and 

influence) 
use proportional reasoning 

This rather overwhelming list is by no means complete, and I'm sure that 
the thoughtful reader can add a few more of his or her favorites. Clearly, no 
single program could possibly achieve all of these goals or even a majority of 
them. Many of these goals are subject-matter specific Identifying the 
assumptions made by a physician diagnosing a disease requires quite different 
background l<nowledge from that needed to Identify the assumptions made by an 
economist advocating wage and price controls. What's more, there is little 
consensus as to how to prioritize this list. Which would ^isn Pick as the five 
most important goals from the list above? Different instructors emphasize 
different goals depending upon their subject matter and philosophy of education, 
A program which successfully meets the needs of one instructor (getting 
students to be skeptical about unsupported assertions, for example), may be of 
little value to another instructor whose goal Is to enhance the abstract 
reasoning skills of his students. 

Thus, the choice of appropriate programs will hin^e upon which goals 
instructors choose to focus on. In addition, programs va-y in terms of the 
specific procedures and methods used to achieve goals as well as the goals 
themselves. The methods employed are generally designed to overcome what are 
perceived to be the obstacles to critical thinking within a specific context. 
This means that In selecting a critical thinking program, an Instructor needs to 
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consider not only what he or she wants students to do. but also why those 
students aren't currently exhibiting the Kind of critical thinking desired. Do 
they lack specific reasoning or problem-solving skills? Do xhey lack adequate 
background knowledge and experience? Do they lack patience and self-dls- 
clpllne? Does the educational system Inadvertently punish the desired behavior 
or reward competing behavior? Do students have beliefs and values which keep 
them from utilizing the skills and knowledge that they have? Do they know 
what Is expected of them? The answers to these and other questions will 
determine the suitability of particular methods and procedures to the Instruc- 
tor's specific situation. 

In summary then, the selection of a program to enhance thinking and 
reasoning skills will depend upon: 1) the strategic choice of goals to be 
achieved and 2) the tactical choice of techniques and methods des.gned to 
overcome the Impediments to critical thinking In a specific context. 
Generic v? - Content-soeclfic Programs 

Attempts to enhance thinking and reasoning skills have generally fallen into 
two categories: 1) stand-alone, general-purpose courses and 2) the incorporation 
of thinking-enhancement programs Into traditional, discipline-specific courses. 
Stand-alone, generic courses have great appeal to faculty and administrators 
who wish to enhance skills without disturbing the status quo too much. Such 
courses can be offered as electlves or required prerequisites without the need 
to alter the traditional curriculum. They can also be taught by resident 
"experts" with specialized training. Unfortunately, such courses cannot provide 
much In the way of background knowledge and content-specific problem-solving 
skills needed to do well in a particular discipline. Positive transfer to other 
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courses Is thus severely limited. In addition, research has shown that students 
taking such classes often do not apply the general skllla they laarn In sub- 
sequent courses because they cannot recognize analogous situations In different 
contexts. Those generic courses which seem to get the best transfer results 
generally limit the number of objectives they pursue and make a special effort 
to use examples from lots of different subject areas In order to point out 
analogous situations where the same skill can be applied. Stand-alone courses 
can also be useful In those cases In which It Is possible to target students who 
have deficiencies or problems that are not content specific (for example, 
students with Inappropriate attitudes or expectations). In general, however, 
white generic courses may Impart skills and knowledge of potential benefit to a 
wide variety of different disciplines, the actual Impact on any particular 
subsequent course tends to be small. 

In contrast, when a thinking-skills program is Incorporated Into a regular, 
content-specific course, the impact is much more dramatic within that discipline. 
Unfortunately, scudents show little inclination or ability to transfer what they 
learn in one discipline to another. What's more, the inclusion of material for 
enhancing thinking skills into a course requires a commensurate reduction in 
the amount of time devoted to traditional content That means that either the 
same breadth of material is covered in less depth (which Is usually thin in 
lower-level courses to begin with), or fewer topics can be covered. Needless to 
say, reducing content runs counter to the twentieth century trend of adding 
more and more topics to the curriculum. Many faculty members and administra- 
tors may worry that cutting back on the number of topics or chapters covered 
in a course may short-change the student However, there Is a good deal of 
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evidence to suggest that the vast majority of material acquired through rote 
memory and cramming will be forgotten quickly, while there Is greater long- 
term retention of concepts which are "discovered** and meaningful. Further- 
more, In a world awash with Information, acquiring new facts and concepts may 
not be as Important as being able to evaluate evidence and ideas. It Is 
essential, however, that courses substituting depth for breadth be viewed t/ 
students, faculty, and administrators alike not simply as cut-down courses with 
less material to cover, but as classes In which the Intellectual challenge has 
been Increased substantially. 

A strong case can be made for developing a coordinated program of 
critical thinking "across the curriculum", by emphasizing thinking and reasoning 
skills In a wide variety of content courses. Such a program would have the 
greatest likelihood of long-term success, but convincing large numbers of 
faculty to voluntarily modify their traditional course content would be difficult, 
and the imposition of such a program from the top down might be viewed as 
infringing upon academic freedom. In addition, many instructors might feel that 
they are not adequately trained or do not have the specialized knowledge 
needed to teach thinking skills to college students. (Shouldn't you need a 
degree in cognitive psychology or logic or something?) Alt of the instances of 
content-specific programs with which I am familiar have been implemented on a 
voluntary, and thus very limited, basis with little or no coordination across 
disciplines. Nonetheless, these individual efforts can be effective; and to the 
extent that Informal, cross-disciplinary cooperation can be achieved, even 
greater benefits can be expected. 
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Selecting Program Characteristics 

Given the difficult choices confronting the Instructor or Institution wishing 
to enhance the thinking and reasoning skills of students, how should one 
proceed In selecting or creating a specific program? I would suggest a three- 
step process: 1) determine your long-term strategic goals, 2) assess the 
needs and characteristics of your students. 3) Identify short-term tactical 
goals, consistent with the findings from steps 1 and 2 above, which can be 
realistically attained In the time alloted. Armed with this Information, the 
educator is In a good position to quickly determine the suitability of program 
characteristics for his or her particular needs. 

1. Determining Long-term Goals. The first step in selecting or developing a 
thinking-skills program is deciding upon the ultimate goals you hope to achieve. 
Look at the list on pages 3 and 4 above and add any goals that you feel were 
omitted. Put a check by those which are most relevant to the intended course 
and then try to rank them In order of importance. Select two or three primary 
goals and perhaps several more secondary goals. Realize that within the short 
span of a semester you won't have time to accomplish everything you would 
like to and determine ahead of time which goals will be sacrificed If necessary. 
You will want to find a program with the same goals and priorities to either 
adopt or serve as a model. 

2. Assessing Student Needs . The second step is to evaluate the thinking 
skills of your students. Most instructors are well aware of what the deficienc- 
ies of their students are but often do not know why those deficiencies exist. 
Achieving one's long-term goals will depend upon selecting the proper methods 
to use with your students which, in turn, depend upon why they are having 
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problems. Below are listed tour major problem areas. Try to find out 
which areas your students are having the most trouble with. (This may require 
friendly Interviews with your students, since many of these problems are 
invisible In the classroom.) 

1. Lack or underdevelopm'int of specific thinking or reasoning skills 
e.g. abstrm reasoning 

logical reasoning 
reasoning by analogy 
metaphorical rearming 
proportional reasoning 
inductive rearoning 
deductive reasoning 
Inferential rearoning 
scientific reasoning 
probabilistic reasoning 
analytical reasoning 
synthetic rearoning 
organizational skills 
cwnparatlve reasoning 
metacognltlve awarenesf^ 

creative ability to imagine alternative explanations 
ability to approach unfamiliar situations systematically 

2. Lack of sufficient background knowledge within the discipline 
e.g. basic problem--solving strategies 

fundamental concepts and vocabulary 
mathematical skills 

different philosophical points of view 

unstated assumptions 

accepted or traditional research methods 

basic factual information 

ruies for evaluating evidence 

guidelines for Judging the credibility of sources 

awareness of important variables 

awareness of important and controversial Issues 

awareness of potential sources of bins 

awareness of potential confounding variables 

reading and writing skills 

awareness of the skills, attitudes and values that are expected 
lack of "hands on" sensory experience 

3. Interfering attitudes and values 

e.g. uncritical acceptance of authoritative pronouncements 
dislike of intellectual confrontation 

dominance of wishful thinking or personal needs over reason 
hostility towards science or education 
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defensiveness resulting from perceived threats to cherished 
beliefs 

perception of a threat to self-esteem 
need for certainty regardless of evidence 
lack of patience and self-discipline to persist in solving 
problMiis 

winning or social expediency considered more important than 

ascertaining the truth 
trust In intuition more than evidence and logic 
unrealistic fear of criticism or lack of self-confidence 
dependence on others to make decisions for him (or her) 
tendency to see all sources of Information as equally credible 

4. Motivational factors 

e.g. critical thinking skills are not sufficiently rewarded in the 
classroom (especially in terms of grades) 
critical thinking skills are not associated with success outside 
the classroom 

there Is fear of possible punishment or rejection from p&ers or 

authority figures for exhibiting critical tninking outside of 

school (or even in other classes) 
behaviors competing with critical thinking are (or have been) 

rewarded more consistently 
critical thinking is not sufficiently modeled either in or 

outside of class 

3. Setting Short-term Goals . Once you have a feel for what the major 
impediments to critical thinking are for your students, you should be sure that 
the program you create or select addresses those problems. It is also Important 
to be aware of the fact that certain skills or knowledge may be prerequisite for 
others. Suppose your long-term goal is to get your students to use problem- 
solving strategies which require inferential reasoning ability, but you discover 
that your students have trouble with that particular skill. You ma/ have to 
defer on your long-term goal and either 1) concentrate on developing the 
prerequisite ability or 2) focus on problem-solving strategies which do no 
require that ability as short-term goals. 

Consider your students as they are not as you think they should be. 
Many instructors overestimate the extent to which students share what they 
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(the instructors) assume to be common knowledge. The explosion In an^unt and 
diversity of information to which students are exposed (much via TV) has led 
to a emaller common baf^ of bacltground Icnowiedge over the last twenty years. 
This makes reasoning by analogy to common experience difficult. We would 
like to assume that our students have basic skills in reading, writing, and 
math, but we can't always count on that either. Hopefully, assessment testing 
and mandator, r emediation will reduce the number of underprepared students. 
Another probiim which faces instructors in many classes is the surprisingly 
large percentage of students {over SOX) who have difficulty with abstract 
concepts and general principles. They are at what Piaget v.'ould call the 
concrete operational stage of thinking. This means that they try to understand 
new situations In terms of real-life sensory experiences. When a connection is 
not apparent, they don't understand. This presents real problems In courses In 
which abstract reasoning is essential. 

Instructors unfamiliar with assessing student characteristics or in doubt 
about the prerequisite knowledge or skills needed to achieve thsir desired goals 
may need to consult someone with experience in these matters before a final 
decision on program selection is made. In the absence of a consultant, there 
are a number of good books which can provide backgrouno information on 
these topics. Two that I would recommend are: The Teaching of Thinking by R. 
S. Nickerson, D. N. Perkins, & E. E. Smith and Teaching Students to Think 
Critically by Chet Meyers. 

Finally, It is important to recognize that there simply may not be enough 
time in a single course to make much progress on skills which take a long 
time on task to develop. Just as reading and writing skills slowly develop 
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with practice, so it is with many thinking and reasoning sttilis. For example) it 
Is unlilcely that many students wilt show significant gains in abstract reasoning 
ability as a result of a single, one-semester course. A coordinated effort over 
several courses and semesters (such as Project ADAPT, described below) can 
produce measurable improvement in complex skills, but individual Instructors 
may never see the fruits of their labor. At this point a certain amount of 
faith in the cumulative effects of training over time is required. If the faculty 
within a discipline can agree that such slowly-developing skills are important, 
then the integration of such skill development into the curriculum should be 
encouraged as much as possible. Even without a coordinated effort, however, 
individual instructors should take heart in belief that their efforts are contribu- 
ting to a long-term benefit to the student, even if there is not a dramatic 
improvement by semester's end. 

Examples of Programs 

The following examples reoresent thumbnail descriptions of some of the 
different kinds* of programs now in use. I iiave tried to identify the major 
goals and techniques employed In these programs as well as characteristics of 
the students Involved (If reported). Although the vast array of different 
programs makes categorization difficult, I have selected examples which fall 
Into three main groups: 1) programs designed to develop complex formal or 
abstract reasoning skills, 2) programs designed to teach the basic component 
skills of thinking, and 3) programs geared to teaching heuristic problem-solving 
techniques. 

A number of programs have been designed to help students develop formal 
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thinking and abstract reasoning skills. Many of these programs are based on 
the theory of cognitive development proposed by Jean Plaget and the concept 
of the "learning cycle". According to Plaget, formal operational thinking 
occurs as a result of 1) the failure of concrete thinking to provide adequate 
understanding of a new situation and 2) the development of new, abstract ways 
of thinking which result from a process of exploration, Invention, and finally 
application to the situation of Interest (the learning cycle). In other words, a 
grasp of abstract, general principles stems from discovering the common 
denominator In a number of specific experiences. 

ADAPT fAccent on the Development of Abstract Processes of Thought) 
This program attempted to Integrate the learning cycle approach Into the cur- 
riculum of several freshman-level courses (anthropology, economics, English, 
history, mathematics, and physics) at the University of Nebraska-Lincoln. The 
students were self-selected, tended to be below average In cognitive develop- 
ment, and were required to take a' I of the ADAPT courses If they chose to 
enroll In the program. An effort was made to coordinate the teaching of 
similar thinking skills across courses. The use of discovery learning from 
concrete situations to more abstract principles was encouraged. Gains In 
formal reasoning ability, conceptual complexity, and critical thinking were 
reported at the end of the year, however, there was little effect on grades in 
the following year. 

DOORS (Development of Operational Reasoning Skills) This program was 
based on the ADAPT program but geared to students at Illinois Central 
Community College identified as being older, returning students, having 
undefined career goals, or being average to below-average in high school 
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achlf-vement- The courses required were similar to those In the ADAPT 
program. A c»n8ldsrable effort was made to coordinate the thinking skills 
taught In the various classes, beginning with basic skills such as observation 
and description and proceeding to more advanced skills such as classification 
and Inference. The DOORS program was later expanded to a consortium of 
seven community colleges called COMPAS. Each school developed a program 
within a coordinated framework but tailored to Its Individual needs. The results 
were mixed. 

SOAR (Stress on Analytical Reasoning) Aimed at science and math students 
at Xavier University, project SOAR employed not only the Learning Cycle 
approach but also the thlnklng-aloud approach of Whlmbey and Uochhead and 
vocabulary building. The program was intended for students similar to those 
described above and included competitive problem solving between groups and a 
low student-teacher ratio (7 to 1). Particular emphasis was placed on control 
of variables, proportional reasoning, probability, combinatorial reasoning, and 
recognizing correlations. Substantial increases were reported in formal reason- 
ing ability, vocabulary, and reading comprehension; however, there was no 
comparison with matched controls. 

DORIS (Development of Reasoning in Science) Aimed at freshman science 
majors at California State University at Fullerton, this program focused on the 
same skills as Project SOAR with the addition of hypothesis testing. The 15 
week course spent 4 weeks each on chemistry, physics and mathematics, and 3 
weeks on earth science. Lessons Included both training In specific reasoning 
skills and content objectives. Results were mixed, varying both 
over time and with the testing instrument used. 
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^^fH^nti^ Prenaration for PhYgi« This program also employed a 
Plagetlan approach to enhance abstract thinking skills for pre-physlcs students 
assessed as underprepared at the University of Alabama at Birmingham. The 
content focused on understanding math functions and symbolic logic. This 
summer course met for 4 hours a day. five days a week for 9 weeks. Techni- 
ques Included paired thinking aloud, playing logic games such as Equations, 
WFF'N Proof, and On Word, and learning problem-solving strategies. Students 
completing the course showed Improvement In ACT math scores, level of 
cognitive development, and subsequent grades compared to controls. 

Although subjective reports from students and teachers In these programs 
are usually positive, objective measures of Improvement have gei.^rally been 
weak. It is difficult to say whether this reflects shortcomings, in the teaching 
techniques employed or merely the fact that jne semester does not provide 
enough time on task to develop co...jlex, abstract reasoning skills, even when 
emphasized In several coui ^es at the same time. 

Man> programs have approached the problem of developing thinking skills 
In teri..s of building up the basic components needed for more complex thinking. 
Some of these programs use the classification schemes such as those devised by 
Bloom and Perry to organize tasks In terms of increasing difficulty and 
abstractness. Others are based on specific theories of intelligence such as 
those of Feuerstein, Guilford, or Sternberg. Unfortunately, most of these 
programs have not been evaluated using matched controls. 

. ?APA fSdence...A Process Approach) In this program 105 "hands on 
"modules were developed to teach eight basic scientific processes: observing, 
using space/time relationships, using numbers, measuring, classifying, com 
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munlcatlng, predicting, and Inferring. Each module has modest goals, but they 
are chained together proceeding from specific and concrete to more general and 
abstract Although Initially developed for K-12 level science classes, the 
approach has been adapted for use with (largely remedial) college students as 
welt. 

BASICS (Building and Applying Strategies fo r Intellectual Competenqles In 
Students) The program focuses on 18 thinking strategies organized into two 
sub-units. The first emphasizes data gathering/retrieval strategies (such as 
observing, recalling, noting similarities and differences) and conceptualizing 
strategies (such as classifying, grouping, and concept differentiation). The 
second looks at Interpretation strategies (such as inferring attributes, causes, 
meaning, and effects, generalizing, anticipating, and making choices), attitude- 
development strategies, and skill-development strategies (Improving proficiency). 

SABLE (Systematic Add roaches to Biological k^bpr^tory Exploration) 

Developed at the University of California, Berkeley, Project SABLE was 
designed specifically to teach the basic skills Involved in the scientific method 
(observation, hypothesis formation, and testing). An explicit model of the 
scientific method was introduced step by step through the use of self-Instruc- 
tional tutorials and computer simulations. Students were required to apply the 
principles they learned by finding the solution to a difficult and unfamiliar 
problem. While control students appeared to be as knowledgeable about the 
scientific method as those given SABLE training, the later students did mud. 
better at applying their knowledge to new problems. 

Many of the most impressive cases of student Improvement are the result 
of teaching specialized, heuristic problem-solving techniques. These approaches 
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seem to be best suited to those students who already have some degree of 
experience In the area, and the application of these skills tends 'to be limited to 
a specific discipline, as mentioned earlier. Nonetheless, they can be ©specially 
valuable for majors within a field. Furthermore, Instructors with limited 
background knowledge about cognitive development or assessment can feel 
comfortable incorporating heuristics Into their content classes, and they can 
Implement these programs without the need to confer with committees or 
coordinate efforts with other instructors. Also, Improvement In student 
performance can be detected more readily and quickly. Instructors don't have 
to take it on faith that their efforts will prove to be worthwhile. 

Guided Design The principal goal of this approach was to give engineering 
students at West Virginia University experience with solving open-ended 
problems as part of a cooperative group effort. Students are expected to gain 
knowledge of the relevant subject matter outside of class. During class the 
teacher plays the roles of guide, prompter, manager, and consultant. Problems 
are presented In ascending order of difficulty and designed to ensure that all 
Important course content is included. Emphasis Is placed on the development of 
good decision-making skills. Group discussion Is supplemented with "instruction- 
feedback" pages In which the views of professionals on the questions at hand 
ca.t be compared the conclusions of the student groups. 

Patterns of Problem-Solving This course was developed at UCLA and 
taught by faculty and graduate assistants in a variety of disciplines. Goals 
included Identifying personal problem-solving styles, presenting tools for problem 
representation and the use of models, pointing out the role of values In problem 
solving, learning to make individual and grouo decisions, and learning to 
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overcome conceptual blocks when dealing with uncertainty. Students from ail 
fields and at all levels have been admitted into the couree, however, the 
problem-solving methods studied leaned toward mathematical approaches such as 
statistical decision theory, information theory, and utility theory. The extent 
to which these general problem-solving approaches are applied by students to 
problems In other courses remains to be determined. 

Schoenfeld's Heuristic Instruction In Mathematics Schoenfeld has atte.-npted 
to explicitly teach students in mathematics at the University of California at 
Berl<eley the heuristic problem-solving techniques employed by experts. Having 
discovered that itnowledge of heuristics does not reliably lead their application 
in new problem situations, Schoenfeld has added the teaching of what he terms 
a "managerial strategy" to his course. Students were first instructed in the use 
of such heuristics as: draw a 'diagram if at all possible; if there is an integer 
parameter, look for an inductive argument; consider arguing by contradiction or 
contrapositive; oonsider a similar problem with fewer variables; try to establish 
subgoals. Afterwards, students were taught a managerial strategy in which they 
proceed stepwise through the phases of analysis of givens and unttnowns, 
development of a broad plan for how to proceed, exploration of simiiar or 
modified problems, implementation of a problem-giving plan, and finally 
verification of the solution. Schoenfeld's students demonstrated impressive 
improvement in mathematical problem-solving sl<ills compared with controls. In 
addition, their perceptions of problems became more like those of experts. 

A Practicum tn Thinking This iO-week, group-oriented course offered at 
the University of Cincinnati focused on developing self-awareness in problem 
solving. While working on self-selected problems of personal intarest, students 
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were asked to evaluate their thinking in terms of working with others, recog- 
nizing assumptions, study skills, memory techniques, setting goals, logical 
inference, and decision making among others. Although students rated them- 
selves as more proficient in these areas at the end of the course, there was 
little objective evidence for a substantial increase in actual skills. 

CoRT (Cognitive Research Trust) This British program is aimed at develop- 
ing what Oe Bono refers to as "lateral thinking" or the ability to generate 
novel Ideas. It consists of 63 35-minute lessons organized into six units: 
Breadth; Organization; Interaction; Creativity; Information and Feeling; and 
Action. It is considered suitable for students from grade school through college 
a%,. In each unit students learn operations which help them to achieve specific 
goals and associate a mnemonic with each operation to help them remember to 
apply them in problem-solving situations. Printed notes and practice exercises 
are presented with each lesson. Interestingly, De Bono recommends that these 
heuristics for creativity be taught in isolation from specific content in order to 
present the ideas in the clearest possible way. Students receiving CoRT 
instruction ware reported to have significantly greater ideational fiuency, 
generated more novel ideas, and showed greater tolerance for the views of 
others. 

A targe number of programs have been developed to facilitate critic^ 
thinking In reading and writing classes. These efforts range from simply 
spending more time on rhetoric to elaborate programs with structured exercises 
designed to teach specific thinking skills. The later would seem to be im- 
portant since learning how to analyze the anatomy of an argument a la 
Toulmin or how to classify different logical fallacies Is not likely to change 
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habitual ways of thinking without practice and application. Unfortunately, 
there has been little effort to evaluate these programs using matched controls, 
and their effectiveness remains largely undetermined. Clearly, reading and 
writing are vehicles for thought, and writing In particular Imposes organization- 
al requirements on thought that verbal discourse does' not. Although many of 
these programs were originally Intended for elementary or high school students, 
they may prove to be valuable at the college level as well, particularly for 
remedial classes. 

Some Final Thoughts 

One of the most disconcerting aspects of my research on efforts to 
enhance thinking skills Is the lack of Impressive, objective success. While 
there are many glowing subjective reports of benefits to both students and 
teachers, with few exceptions, In comparison to matched controls, these 
programs have had little Impact on objective test scores or subsequent grades. 
Are the subjective reports to be taken as cases of self-deception or placebo 
effects? In some cases, perhaps, but in general I think not. Why 
then don't measurable Improvements in specific skills result in improved 
grades? Three possibilities come to mind. 

1. A specific thinking skill may be necessary but not sufficient for solving 
a particular problem or the understanding of a complex idea. Prerequisite 
knowledge, additional reasoning skills, recognition of context, persistence, and 
other factors may be required as well. Testing for particular skills may not 
reveal other deficiencies that impair performance in class. 

2. Improvement in critical thinking skills may not be reflected in course 
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grades by virtue of the fact that those skltle are not graded. If the tests 
used to determine grades are based largely recall of simple facts and vocabu- 
lary, for example, Improved ability to challenge assumptions, make logical 
Inferences, examine evidence, etc would remain Invisible. Thare Is a strong 
tendency for teachers to adjust course content and evaluation measures to 
permit at least a majority of their student to succeed. If, as research In- 
dicates, the majority of students have difficulty with abstract, Inferential 
reasoning; Instructors may base only small percentage of the course grade on 
performance which requires those skills. 

3. After decades of trying, educators have yet to Identify any one skill, 
aptitude, or personality trait that predicts the majority of the variance In 
school grades, why not? Because a major factor Influencing grades cannot be 
adequately measured by tests for skills and traits: motivation. Students with 
below-average skills can often succeed by Investing more time and effort In the 
rote memory of facts and vocabulary for Instance. Alternately, a student with 
above-average skills may not bother to 

apply them or prefer to spend time In other than academic pursuits. A student 
may admire or despise his teacher, may like or dislike the subject matter, may 
make fiends or enemies with other students in the class, and grades can be 
affected by the differences in attitudes and behavior that result. Because 
motivation is not perfectly correlated with reasoning skills, the correspondence 
between skiits and grades in reduced. 

Any or ail of these three factors may Influence outcome assessment in a 
particular situation. Unle'=;s some means of measuring or controlling these 
variables can be found, outcome assessment research will inevitably be plagued 
by a high degree of uncertainty. 
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